Crossing symmetry - #49
Conversation
… driver Crossing symmetry lets one generated matrix element evaluate physically related processes by moving legs between the initial and final state, encoded in an extended flavor index (id = cross*nmaxflavor + flavor). - Fortran, C++ and mg7 (madmatrix) exporters emit the crossing tables, the per-event momentum/NSF permutation, the crossed denominator (initial spin*color x identical-final-state factor) and the sigma-permuted good-helicity remap / union. - `check crossing <proc> [--exporter=standalone|standalone_cpp|standalone_mg7] [--simd=...]` matches each reference subprocess against the crossing of the collapsed output that reproduces it; help text and auto-completion updated, and the "not checked" message now distinguishes its three causes. - Fix a stack buffer overflow in the mg7 UMAMI SIMD flavor-sorting path: it grouped events into SIMD vectors by the raw extended flavor id into arrays sized nmaxflavor, overflowing them for any crossing (id >= nmaxflavor) and crashing check_sa.exe (SIGABRT/SIGSEGV). Group by the reduced flavor instead and write flavor_indices per event, padding unused lanes with a valid crossing-0 id so the per-event momentum gather never indexes the crossing tables out of range. Validated: `check p p > j j j --exporter=standalone_mg7` now reports 1161/1161 flavor/crossing subprocesses passing with zero "not checked" (g d > d d d~ and g d > d u u~ previously crashed); the standalone cross-symmetry acceptance tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…pleteness) --use_crossing=False mapped to merge_crossing=True, which takes the "do not generate diagrams" branch in generate_diagrams_multiprocess and drops the crossed subprocesses from the amplitude list entirely -- an incomplete partonic sum (verified: p p > j j gave 3 of 8 subprocesses, missing g q > g q and q q~ > g g). In 3.x merge_crossing defaulted to False (crossed processes kept as cross_amplitudes); only the old --no_crossing debug flag dropped them. Decouple the two: keep crossed subprocesses unconditionally (merge_crossing=False). --use_crossing now only decides, at the exporter stage (via self._use_crossing), whether they collapse into a single extended-FLAV_IDX matrix element (fortran standalone) or are written out as their own matrix elements (every other output, incl. madevent). --use_crossing=True generation is byte-identical (already mapped to False); only --use_crossing=False changes, and it is now a complete output again -- the fallback the madevent gate already points users to. Verified: p p > j j --use_crossing=False -> output madevent now yields all 8 subprocesses (5 P dirs); test_standalone_cross_symmetry 36/36 OK. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…te-equivalent Scaffold the crossing-aware evaluation path in the madevent group matrix template without changing behaviour yet. Add holes to the SMATRIX/MATRIX of matrix_madevent_group_v4.inc: - smatrix_me_cross_decl / smatrix_me_cross_decode (decode IFLAV -> CROSS, FLAV_USE and APPLY_CROSSING to build crossed P/NHEL/IC), - me_flav_key token (IFLAV off / FLAV_USE on) at every GOODHEL/NTRY/GET_FLAVOR, - me_matrix_args for both MATRIX call sites, - smatrix_me_iden_line (crossed denominator branch), - me_matrix_ic_param / me_matrix_ic_decl (runtime IC into MATRIX), - crossing_routines_me + smatrix_me_goodhel_or. fill_crossing_replace_dict_me() fills them; its OFF branch reproduces the historical madevent code and the ON branch is added in slice 2. The madevent exporter calls it with use_crossing=False for now, so the output is functionally identical (a p p > j j regeneration differs only by two comment lines and three blank lines). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…es+links) Implement the crossing-on fills of fill_crossing_replace_dict_me: decode the extended FLAV_IDX into (CROSS, FLAV_USE), short-circuit an unusable crossing (spin*color = 0) to a zero ME, and build the crossed P/NHEL/IC once via APPLY_CROSSING_TABLE before the helicity loop; MATRIX now takes a runtime IC and its helas calls read <base_nsf>*IC(i) (use_crossing_ic). The denominator uses GET_SPINCOL_CROSS*GET_IDENT_CROSS for a genuine crossing and the historical IDEN*BROKEN_SYM for CROSS=0. The crossing routines are emitted with a per-proc qualifier (CR<pid>_GET_CROSS_PERM, ...) so the matrix<i>.f of one group do not clash at link time. The good-helicity filter is bypassed for a crossing-enabled ME (a crossing permutes/flips helicities; a shared-flavor remap can optimise this later). write_matrix_element_v4 computes me_use_crossing (opt use_crossing, group template only, and the process definition must not pin an s-channel) and drives use_crossing_ic from it. The madevent crossing gate is unchanged, so this stays dormant until the per-crossing auto_dsig/metadata land: a normal --use_crossing=False output is byte-identical, and the ON matrix element (matrix1_orig) compiles and links (madevent_forhel) for p p > j j. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Helicity recycling (hel_recycle.py) rewrites matrix_orig.f into a single sweep with fixed, baked-in helicity values. That cannot coexist with the runtime helicity permutation a crossing applies: one baked helicity set cannot serve every crossing of a merged matrix element, and the _hel template MATRIX has no runtime IC to thread. So the group exporter now turns helicity recycling off when the output is written with crossing on, and compiles the crossing-aware matrix<i>.f directly. Verified: with crossing on the group emits matrix<i>.f (no _orig/template pair) and the default `make madevent` target links for p p > j j; a normal --use_crossing=False output is unchanged (recycling still on, matrix<i>_orig.f + template_matrix<i>.f). Gated on opt use_crossing, so it is inert until the madevent crossing gate is opened. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Helicity recycling IS compatible with crossing, via the same permutation that lets goodhel be shared. hel_recycle.py rewrites only the helicity argument of an external call (NHEL(k) -> a baked +-1) and copies the momentum slot and the NSF argument unchanged, so an IC-aware matrix_orig.f yields IC-aware baked calls (e.g. IXXXXX(P(0,1),ZERO,+1,+1*IC(1),...)). The recycled MATRIX loops over the base good-helicity set; feeding it the crossed momenta PUSE and IC evaluates that set at the crossed kinematics, which by H=sigma(K) is exactly the crossed matrix element -- no NHEL table nor an explicit helicity remap is needed in the recycled code. So: revert the "disable helicity recycling for a crossing output" stopgap, and give matrix_madevent_group_v4_hel.inc the same crossing holes as the non-hel template (decode + PUSE/IC build, crossed denominator, IC into MATRIX, per-proc CR<pid>_ routines), minus NHELUSE. fill_crossing_replace_dict_me fills the _hel variants (smatrix_hel_cross_decl/decode, hel_matrix_call_args, hel_matrix_ic_param). Verified (crossing gate bypassed in a throwaway harness): with recycling ON the group emits matrix1_orig.f + template_matrix1.f (both crossing-aware, no leftover holes); madevent_forhel links; and running HelicityRecycler on them produces a matrix1_optim.f whose baked calls keep *IC(k) and which compiles+links into the full madevent. OFF (--use_crossing=False) output unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add ProcessExporterFortran.partition_crossing_classes(matrix_elements): group a subprocess group's matrix elements into crossing-equivalence classes. A member is tied to a base when the base's crossing enumeration (compute_crossing_pdg_entries) reproduces the member's crossed physical-PDG signature -- the same key check_crossing matches on -- returning per member a (base_index, cross) pair. This is the basis for sharing one matrix<i>.f across a base and its crossings (the base SMATRIX, driven by an extended FLAV_IDX, evaluates the whole class), which the exporter wiring will use next. Pure function, not wired into the exporter yet. Unit test TestCrossingPartition on p p > j j: within P_gq_gq, g q~ > g q~ is found as a crossing of g q > g q (2 MEs -> 1 base); within P_qq_qq, q~ q~ > q~ q~ crosses q q > q q (3 -> 2), while q q~ > q q~ stays its own base. Cross-*group* crossings (g g > q q~ vs q q~ > g g, separate P dirs) are not merged -- that needs crossing-aware grouping, a later step. test_standalone_cross_symmetry: 37/37 OK. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The previous ME-level partition was wrong: it matched a module by its first flavor's signature only, so it silently missed genuine crossings (e.g. it claimed u u~ > u u~ is not a crossing of u u > u u, when it is exactly cross=4). More fundamentally, the crossing relates flavor combinations, not whole modules: a flavor-merged module bundles flavors that cross to different bases (the Q Q~ > Q Q~ module carries both u u~ > u u~, a crossing of Q Q > Q Q, and d d~ > u u~, which is not), so an ME-level base/cross map cannot be right. Rework the helper to route per flavor: a module may drop its own matrix<i>.f only when EVERY one of its flavors is a genuine crossing of some base module's flavor; otherwise it stays a base. Returns (bases, routing), routing[i] giving one (base_index, iflav) per flavor -- the base SMATRIX and the 1-based extended FLAV_IDX (cross*nflav+flav) to reach that flavor. This is what per-flavor auto_dsig routing (piece 2) needs. Verified on p p > j j: gq_gq g q~ > g q~ routes to g q > g q (cross 4), 2 MEs -> 1 base; qq_qq q~ q~ > q~ q~ routes to q q > q q (cross 19) and is eliminated, while Q Q~ > Q Q~ stays a base because of its d d~ > u u~ flavor. TestCrossingPartition updated; suite green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Verifying the n_flav/NFLAV consistency (needed before the per-flavor auto_dsig routing) turned up a real M0 bug. fill_crossing_replace_dict_me sized the extended-FLAV_IDX NFLAV from _build_flav_table_flat() (compute_flavor_masks, the STANDALONE convention), but the madevent GET_FLAVOR table is sized by get_external_flavors_with_iden() (== replace_dict 'max_flavor', what FLAVOR(NEXTERNAL,max_flavor)/MAXFLAVPERPROC use). For a merged group ME the two differ -- e.g. Q Q~ > g g has iden 1 but masks 4 -- so NFLAV=4 while the FLAVOR table has a single row, and FLAV_USE=mod(IFLAV-1,4)+1 would index it out of bounds at run time. Use get_external_flavors_with_iden() for NFLAV. This also makes the madevent decode agree with compute_crossing_pdg_entries (which already uses iden), so partition_crossing_classes' routed FLAV_IDX decodes correctly in the base SMATRIX -- the property the per-flavor routing (piece 2) relies on. Verified on p p > j j (crossing on): every matrix<i>_orig.f now has PARAMETER(NFLAV) equal to its FLAVOR(NEXTERNAL,rows) count (0 mismatches over 8 matrices, incl. the NFLAV=2 module), and forhel still links. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…e ME Wire the crossing merge into the madevent group output. generate_subprocess_ directory now partitions the group's matrix elements (partition_crossing_ classes) and, for a subprocess whose every flavor is a crossing of a base subprocess's flavor, writes a light router matrix<i>.f instead of the full one: it keeps GET_FLAVOR<i> (for the PDF) and a router SMATRIX<i> that dispatches per flavor to the base SMATRIX with the crossed FLAV_IDX, dropping the heavy MATRIX<i>/helas. The base SMATRIX (M0) crosses the momenta and rebuilds the crossed denominator, so the routed ANS is the subprocess's matrix element. get_nhel<i> lives in auto_dsig<i>.f so it is unaffected; the auto_dsig itself is unchanged (it still calls SMATRIX<i>/GET_FLAVOR<i>). - New template matrix_madevent_group_router_v4.inc; write_matrix_router_file. - ProcessExporterFortranMEGroup.supports_crossing = True (the _check_crossing_ support gate now lets a grouped madevent output through with --use_crossing). - Helicity recycling is turned off for a merged group for now (the router writes a static matrix<i>.f while recycling builds matrix<i>_optim.f at run time and the makefile globs one or the other; mixing them needs build-system work). - gen_infohtml.get_diagram_nb: tolerate a matrix<i>.f with no diagram comment (a router shares the base's diagrams). Validated: p p > j j, low-stat integration. use_crossing=True (merged/router) gives 5.449e8 +- 2.8e6 pb vs use_crossing=False (complete reference) 5.448e8 +- 2.8e6 pb -- agree within MC error. P1_qq_qq runs 3 subprocesses on 2 heavy MEs, P1_gq_gq 2 on 1; both link and integrate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… disable) Let a merged crossing group keep helicity recycling on its heavy base matrix elements while the light routers stay static. A router now writes matrix<i>_router.f (not matrix<i>.f): the makefile globs matrix*_router.o into both the forhel and the optimized binaries, gen_ximprove recycles only matrix*orig.f so it leaves routers untouched, and a base is recycled exactly as before (matrix<b>_orig.f + template -> matrix<b>_optim.f). Removes the stopgap that disabled recycling for a whole merged group. gen_infohtml.get_diagram_nb also looks for matrix<i>_router.f and returns 0 when a subprocess has no matrix file of its own (a router shares the base's diagrams). Validated p p > j j, hel_recycling=True: bases produce matrix<b>_optim.f, the router matrix3_router.o compiles and links, the full forhel->recycle->madevent run completes and gives 5.468e8 +- 3.0e6 pb vs the use_crossing=False reference 5.448e8 +- 2.8e6 pb (agree in MC error). Also validated (earlier commits' merge path): p p > t t~ j j merged == reference 344.8 pb exactly (massive tops), and p p > j j j merged 4.912e7 vs reference 4.923e7 pb (agree in MC error). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
TestCrossingUnsupportedOutput asserted that a madevent output refuses a process generated with crossing. That was true before Track A; the grouped madevent exporter now supports crossing (supports_crossing=True, the crossing router shares a base matrix element), so move 'madevent' out of UNSUPPORTED_FORMATS (matchbox stays) and assert instead that both standalone and madevent accept it. Verified: TestCrossingUnsupportedOutput 3/3 OK. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Checking the LHE colour flow (thanks to the reviewer's prompt) turned up a real bug. A router returns the base's selected colour-flow index ICOL, but events are written through the router's own leshouche ICOLUP. For p p > j j the two flow orders happen to coincide, but in general the crossed colour reps decompose the shared colour basis in a DIFFERENT order (verified: for p p > j j j and p p > t t~ j j the router flow set equals the crossed base set but is permuted), so ICOLUP(ICOL) is a valid but WRONG flow -- colour conservation still holds, so it is invisible in the event, but the parton shower would get the wrong colour connection. Add a per-flavor COLMAP in the router: match each crossed base flow (leg j <- base flow leg perm^-1(j), colour<->anticolour where that leg swapped initial/final) to the local flow of the same topology, and remap ICOL after the base call. Emitted only when non-identity. Helicity needs no such map: the router's own get_nhel already enumerates the crossed helicities in the base's order (rh[hb] = bh[hb] with the crossed legs sign-flipped, verified for every router in p p > j j / j j j). Verified: p p > j j j routers now carry COLMAP /4,3,2,1/ and /3,4,1,2/, compile, and the full run gives 4.929e7 +- 3.3e5 pb vs reference 4.923e7 +- 4.2e5 (xsec unchanged, as ICOL only labels the flow) with colour conserved in all events. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Crossing reuses one matrix element across physically distinct (crossed) initial states, so a per-beam property is ill-defined. Rather than return silently-wrong numbers, tag the output and raise a clear error at run time. Generation: in generate_subprocess_directory, append 'crossing' to proc_characteristic['limitations'] -- but only when crossing is materially applied (a router is emitted, or a base evaluates a cross>0 flavor). A process where crossing does nothing (e.g. two unrelated single-process groups) stays untagged and is never blocked. Run time: in check_card_consistency (LO branch, next to the dressed_ee check), when 'crossing' is tagged, raise InvalidCmd if polbeam1/polbeam2 != 0 (beam polarisation) or pdlabel/pdlabel1/pdlabel2 == 'eva' (EVA luminosity). The message points to 'generate <process> --use_crossing=False'. The guard sits inside the existing RunCardLO gate, so NLO cards (no polbeam) are untouched. Verified: p p > j j tags ['crossing'] (round-trips as a list); the guard blocks polbeam=-100 and pdlabel=eva, and does NOT block when limitations=[] with the same settings; --use_crossing=False gives limitations=[], 0 routers, 8 full matrix.f (falls back to the 3.x per-subprocess matrix elements). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…hoton) For lepton/photon beams (define EP=e+ a; EM=e- a; generate EP EM > ...) each initial state lands in its own single-process P directory and the crossings relate DIFFERENT directories, so the within-group router (Track A) never fires and every group compiles its own matrix element even when several are crossings of one base. This adds cross-group reuse: a dependent group symlinks the base group's crossing-aware SMATRIX and routes to it, keeping its own phase space and PDFs. - compute_crossgroup_routing(): flatten every group's matrix elements, run the (group-agnostic) crossing partition, and return, per dependent, the base directory/proc_id and the crossed FLAV_IDX per flavor. It bows out entirely if ANY group has within-group routing -- that is the hadronic p p case, which Track A owns and where cross-group merging is a separate deferred optimisation. Wired into export_processes before the group loop. - generate_subprocess_directory: a dependent writes NO matrix element of its own; it symlinks the base group's matrix<b>.f (+ template with hel recycling) and its auto_dsig routes to the base SMATRIX. - auto_dsig_v4.inc: backward-compatible holes (default == original) for the flavor lookup and the SMATRIX call in BOTH the scalar and vectorised (SMATRIX_MULTI) paths, plus per-program-unit declaration holes. A dependent inlines its own beams (DSIG_XGFLAV, since it cannot own a GET_FLAVOR without clashing with the symlinked base's) and calls SMATRIX<base>(P, DSIG_XGROUTE (IFLAV), ...). Verified p p > j j auto_dsig is BYTE-IDENTICAL. Validated on EP EM > EP EM: check crossing 5/5 at machine precision; the dependent compiles and links; and a full generate_events (direct e+e- collider) gives 43.23 +- 0.17 pb (crossing) vs 43.40 +- 0.16 pb (independent) with every subprocess -- all three cross-group dependents included -- agreeing. Deferred: sharing the compiled .o across directories (recompilation is still per-dir), COLMAP/hel-map for the LHE event records, CONFIGMAP for multi-channel at higher multiplicity, and tagging the 'crossing' limitation for the cross- group case. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…P dirs Cross-group dependents symlinked the base group's matrix-element SOURCE but each directory still recompiled it. Reuse the compiled object instead, and build every P directory in one parallel call. - write_crossgroup_mk(): each dependent P dir gets a crossgroup.mk (pulled in by the shared makefile via `-include crossgroup.mk`, a no-op where absent) whose specific rule symlinks matrix<b>_orig.o from the base directory rather than recompiling the symlinked source; the base object is built first (the specific rule overrides the %.o:%.f pattern, and a recursive rule is the standalone ordering fallback). Only matrix<b>_orig.o -- the full matrix element, identical across the crossing class -- is shared; matrix<b>_optim.o is NOT, because gen_ximprove bakes each subprocess's own good-helicity set into it (it is subprocess-specific). Without recycling the single matrix<b>.o is shared. - write_crossgroup_parallel_makefile(): SubProcesses/makefile_madevent builds every P directory with `make -f makefile_madevent -jN` (or `... forhel`), delegating each to its own makefile and adding `<dep>/madevent: <base>/madevent` ordering so make compiles the base before its dependents while running the rest in parallel. Verified: parallel forhel build (-j4) links all binaries with the dependents' matrix1_orig.o symlinked to the base (compiled once); a full generate_events still gives 43.23 pb; p p is unaffected (no crossgroup files, auto_dsig byte-identical, the -include is a no-op); acceptance suite 37/37 OK. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…roup A cross-group dependent evaluates its matrix element through the base group's SMATRIX, which selects a helicity/colour in the BASE's enumeration; but the event is written through this subprocess's OWN get_helicities / ICOLUP, so the index must be translated. Without it the LHE events carried the base's helicity labels against the dependent's table -- e.g. e+ e- > a a events showed unphysical same-helicity e+e- pairs (the cross section, summed over helicities, was already right, so this was invisible until the events were inspected). - _crossgroup_helmap(dep_me, base_me, cross): 1-based map from a base helicity index to the dependent helicity index carrying the physically-crossed configuration. The base APPLY_CROSSING permutes NHEL by PERM and flips it by the IC sign, so dep config[k] = base_row[PERM[k]]*SGN[k]; the result is a clean permutation of the dependent's helicity table. - _dsig_crossgroup_fills emits per-flavor DSIG_XGHEL (and DSIG_XGCOL for colour, which is the identity and thus skipped for colourless lepton/photon) and applies selected_hel = DSIG_XGHEL(selected_hel, IFLAV) after the base SMATRIX call in both the scalar and vectorised (SMATRIX_MULTI) paths. Emitted only when non-identity; the new dsig_smatrix_vec_post hole attaches to the call's closing paren so the default is byte-identical. Verified: e+ e- > a a events now have physical opposite-helicity e+e- (matching the independent generation, zero same-helicity events); xsec 43.32 pb; p p auto_dsig byte-identical; acceptance suite 37/37 OK. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The base group's compiled orig object was already shared, but each dependent
still compiled its own _optim: helicity recycling bakes each subprocess's good-
helicity INDEX set, and under crossing those sets are permutations of each other,
so the base's baked set does not cover a dependent's. The fix mirrors what the
within-group case gets for free (there the base's forhel survey already sees all
the crossed flavors): bake the base optim over the UNION good-hel of the crossing
class. Because G_dep = pi^-1(G_base), the union is derivable from the base's set
plus the crossing permutations -- no dependent surveys needed.
- _crossgroup_base_helperm(base_me, cross): the base->base helicity permutation
pi[hb] = the base index whose NHEL row equals the crossed row of hb (factored
_crossed_helicity_configs, shared with _crossgroup_helmap). The dependent for
that crossing is good at h iff pi[h] is good for the base.
- export persists these per base directory in crossgroup_helunion.dat.
- gen_ximprove reads it and, before hel_recycle, expands the base's good-hel to
the union.
- crossgroup.mk re-adds the matrix<b>_optim.o symlink. The `-include crossgroup.mk`
moves to the END of the SubProcesses makefile so its specific rules override the
$(MATRIX) static-pattern rule for the optim objects (orig already worked from the
top since orig is not in $(MATRIX)).
Verified on EP EM > EP EM: base optim now bakes the union {3,5,8,9,12,14} (6, was
4); all three dependents' matrix1_orig.o AND matrix1_optim.o are symlinks to the
base (compiled once); xsec 43.32 pb; e+ e- > a a event helicities still physical;
p p matrix+auto_dsig byte-identical with no crossgroup files; acceptance suite
37/37 OK.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The within-group case already tags the 'crossing' limitation so check_card_consistency blocks beam polarisation / EVA (a per-beam property is ill-defined once one matrix element is shared across crossed initial states). Cross-group (lepton/photon) reuse shares a matrix element across even more distinct initial states (e+ vs e- vs gamma), so tag it the same way when compute_crossgroup_routing returns any routing. Verified: EP EM > EP EM now reports limitations = ['crossing'] and the guard blocks polbeam/EVA; the non-crossing EP EM > mu+ mu- (two unrelated groups) stays []; p p > j j still tags via the within-group path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A cross-group dependent samples its own config's poles in genps, but the base SMATRIX enhances AMP2(channel) in the BASE's diagram numbering, so the channel must name the base diagram of the same topology -- otherwise the importance sampling is mis-paired. This never changes the result (summing the channels gives the full integral for any bijective pairing), only the convergence, which matters at higher multiplicity. - _diagram_leg_subsets(me): per diagram, the set of its internal propagators' canonical external-leg subsets (from get_s_and_t_channels; propagators are the negative leg numbers, the trailing single-external-leg t-channel one dropped; a subset and its complement are one propagator) -- a crossing-covariant topology signature. - _crossgroup_configmap(dep, base, cross): map each dependent diagram's subsets through the crossing leg permutation and match to the base diagram with that signature; identity if not a clean permutation. - _dsig_crossgroup_fills emits DSIG_XGCONFIG and wraps the channel as DSIG_XGCONFIG(channel, IFLAV) in both the scalar call and the new vec channel hole, only when non-identity. Verified on EP EM > EP EM: a e > a e is the identity (absent), but a e- > a e- and e+ e- > a a get /2,1/ (identity was mis-pairing the two diagrams there); xsec 43.32 pb; p p matrix+auto_dsig byte-identical; acceptance suite 37/37 OK. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cross-group reuse is the same engine as the lepton/photon case; it was gated off for p p only to protect the validated Track A. Relax the gate from all-or-nothing (bail out if ANY group has within-group routing) to a per-group exclusion: a group is a cross-group candidate only when every one of its members is a within-group base (no router). Multi-ME p p groups (the `j`-multiparticle ones with within-group routers) are skipped and stay Track A; the single-crossing-class groups now also share across P directories -- e.g. q q~ > g g reuses g g > q q~. Validated p p > j j: detection routes P1_qq_gg -> P1_gg_qq; the dependent carries all four maps, including the first non-trivial COLOURED COLMAP (/2,1/), CONFIGMAP (/1,3,2/), the hel-map and the union optim; its matrix1_orig.o and matrix1_optim.o symlink the base (compiled once). Cross section 6.974e8 pb, bit-identical to the --use_crossing=False reference; colour conserved in 1000/1000 events (both); Track A multi-ME groups byte-identical to before the degate; acceptance suite 37/37 OK. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Outside the complex-mass scheme a spacelike (t-channel, P^2<0) propagator has no pole to regulate, so keeping the width i*M*Gamma in the denominator is spurious (it breaks gauge cancellations). The ALOHA propagator routine now tests the sign of P^2 at runtime and drops the width for spacelike momenta, keeping it for timelike (s-channel) ones. This replaces the old code-generation mechanism (helas_call_writers rewrote the call to pass ZERO as the width for is_t_channel() wavefunctions). That was topology-based and matched only madevent's fk_ widths, so standalone / standalone_cpp (which pass MDL_ widths) never got the treatment; doing it inside ALOHA applies it consistently to every tree-level backend. The existing zerowidth_tchannel option (default True) now drives the new aloha.t_channel_width flag instead of the call-rewrite; True keeps the proper treatment, False restores the width everywhere. It is a generation-time (output) option, so setting it at madevent run time now raises a clear error. Fortran/C++/Python (and GPU via C++) writers updated; ignored under the complex-mass scheme. IOTest references for the changed propagator routines still need regenerating. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…icity recycling Two correctness fixes for cross-group (Track B) crossing, where a dependent subprocess routes to a base group's symlinked, crossing-aware matrix element. 1. Colour selection. The base SMATRIX picked the event colour flow with select_color, which masks the base-order JAMP2 with the DEPENDENT binary's ICOLAMP + ICONFIG (mismatched in flow order AND config space); the picked flow could be incompatible with the sampled config and addmothers then failed to reduce its ICOLUP (a p p > t t~ j j refine crash). The base now publishes its per-flow JAMP2 (COMMON/TO_XG_JAMP2) and the dependent permutes it into its own flow order and runs its own SELECT_COLOR (the XG_SELCOL helper) -- a native colour selection. Emitted only for MEs that are cross-group bases, so every other madevent ME stays byte-identical. 2. Helicity recycling. The recycled optim bakes the helicity configs into its wavefunction calls and takes no runtime NHEL, but a crossed dependent is evaluated with a permuted NHELUSE from APPLY_CROSSING. The full helicity sum is invariant under that permutation, but the optim's base good-hel SUBSET is not the dependent's, so its partial sum was over the wrong configs (~2% low for e.g. photon-lepton Compton crossings). A cross-group base now keeps ALL helicity configs in the shared optim (exact for every crossing), still recycling wavefunctions. Validated: p p > t t~ j j completes, total within MC error, colour conserved; lepton/photon EP EM > EP EM crossing now matches the non-crossing reference (was ~1.8% low); p p > j j and p p > j j j unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ra NSF sign) The LHE event helicity (SPINUP, unwgt.f jpart(7,i)=nhel(i)) is the raw NHEL table value, never NHEL*IC. APPLY_CROSSING permutes NHEL but flips only the IC/NSF flags, so the correct crossed label for leg k is base_row[PERM[k]] with NO extra sign: the base MATRIX gives leg k physical helicity NHEL(k)*IC(k)=base_row[PERM[k]]*SGN[k]*IC_IN[PERM[k]]=base_row[PERM[k]]*IC_dep[k], matching the dependent's native label iff NHEL_dep[k]=base_row[PERM[k]]. Both event-helicity maps were multiplying in SGN, double-counting the crossing flip and flipping every fermion/vector leg that swaps initial<->final. Invisible to xsec/colour/flavour (all helicity-summed) and to non-chiral p p > j j (where the per-leg density is (++)==(--)), but wrong for chiral finals: on p p > w+ w- j j the incoming/outgoing antiquark helicities came out fully flipped (~34 sigma). Two maps carried the bug, both fixed by using the UNSIGNED crossed config: - Track B cross-group dependent: _crossgroup_helmap (DSIG_XGHEL). Added a `signed` flag to _crossed_helicity_configs; the helmap now passes signed=False. _crossgroup_base_helperm keeps signed=True -- its good-hel-set remap IS the GHREMAP sigma (table-space, validated by _GOODHEL_PROBE), which needs the sign. - Track A within-group router: write_matrix_router_file returned the base's selected IHEL straight through (COLMAP was applied to ICOL, nothing to IHEL). It now applies the same unsigned _crossgroup_helmap to IHEL, mirroring COLMAP. Validated: p p > w+ w- j j crossing vs --use_crossing=False (10k evt) -> every quark/antiquark/W/gluon helicity bin consistent (max |pull| 1.2 sigma, was 34); the discriminating chiral standalone density matrix (u d~ > w+ g crossed to u g > w+ d, the crossed d 100%-polarised) matches native per-helicity to machine precision via both the compiled Fortran GET_DENSITY_IDX and the f2py PY_GET_DENSITY_IDX wrapper (2 new acceptance tests). xsec unchanged (labels do not affect |M|^2); goodhel/GHREMAP tests green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Do we need any new functionality for that on the madspace side or will it just affect the generation of the subprocesses.json file and we will have a few more flavors within each subprocess? |
|
The things that was needed for madevent were multiple map to correctly connect the information
So yes only subprocesses.json need to be modified. I forwarded your question to claude: Short answer: It's not just "a few more flavors within each subprocess" — a crossing changes the initial state, so a crossed process can't live inside an existing subprocess's flavor list. But the extra work on the madspace side is small and localized: for the cross-section it's essentially just "pass the extended flavor id to the shared ME"; the phase-space machinery is untouched. The only genuinely new piece is for event generation (the per-crossing helicity/colour/channel index maps), and that's a design choice about where those tables live. Why it isn't just more flavors in a subprocess. In subprocesses.json each subprocess has a fixed incoming/outgoing, and its flavors are what the exporter calls all_flavors_same_initial — different PDG assignments that share the same initial-state slots, hence the same kinematics/phase space. A crossing by definition moves a leg between the initial and final state, so a crossed process has a different incoming/outgoing (different PDF legs, generally different masses). It violates the same-initial invariant, so it must be its own subprocess entry, not an extra entry in the base's flavors list. What actually changes in subprocesses.json. You get additional subprocess entries (the crossed ones), each with its own incoming/outgoing/channels/helicities and its own phase space, but pointing at the same me_path (shared compiled ME) plus one new field: the crossing / extended flavor id (cross*nflav + flav). This mirrors the Fortran madevent design exactly — each crossed process keeps its own phase-space setup and only the matrix-element object is shared. What madspace needs — split by use case: Cross-section / integration: essentially nothing new. Each crossed subprocess is an ordinary subprocess with its own incoming/outgoing, so phase space, PDFs, cuts, and channel sampling all work as today. The one requirement is that the ME call forwards the extended flavor id so the ME applies its internal crossing. The madmatrix/cudacpp backend already supports this (supports_crossing=True, extended flavor id) and returns the correct crossed |M|² internally — so no new PS math and no correctness work for the xsec. Event generation (spin/colour in the event record): here there is something. The ME returns the selected helicity, colour flow, and multichannel channel in the base process's enumeration, but the event for the crossed process needs them in the crossed process's own convention. That's three small per-crossing maps — the direct analogues of the Fortran DSIG_XGHEL (helicity), COLMAP (colour), and CONFIGMAP (channel↔diagram). Decision to make: either emit them into subprocesses.json per crossed subprocess and have madspace apply them after the ME call, or bake them into the C++ ME. (Note CONFIGMAP is only an importance-sampling/variance matter, not correctness.) Confirmed: the madmatrix/cudacpp ME currently returns the selected helicity, colour, and channel/iconfig in the base process's numbering — only the |M|² is crossed. So the crossed-process index maps (helicity/colour/channel, = Fortran DSIG_XGHEL/COLMAP/CONFIGMAP) are not yet handled by the ME. For cross-sections that's irrelevant; for event generation someone has to apply them — cleanest inside the C++ ME (the colour one especially, since it needs the base ME to publish its per-flow jamp2 and re-select in the dependent's colour basis), otherwise emitted into subprocesses.json and applied by madspace. |
…decoder
Represent a helicity configuration as a single mixed-radix "canonical code"
over the per-leg helicity states (STATES/NHSTATE), with the last external leg
as the least-significant digit -- matching get_helicity_matrix()'s
itertools.product order, so for a non-polarized process the code of row i is
exactly i (nothing is relabelled). A polarization restriction ({0}/{L}/...)
keeps the full per-leg multiplicity as the radix (helicity 0 / longitudinal
stays a first-class state) and leaves the allowed-code list HELALLOW as the
selected, non-contiguous subset.
matrix_standalone_v4.inc: drop the explicit NHEL config DATA table; add
DECODE_HEL / ENCODE_HEL / FILL_NHEL. PROCESS_NHEL is now materialized at
runtime by decoding HELALLOW (FILL_NHEL, called from SMATRIX / GET_NHEL /
GET_DENSITY_IDX), keeping the density-matrix and f2py interfaces intact. The
external helicity label (USERHEL) is the canonical code.
f2py: the get_nhel_entry accessor (all_matrix.f) now fills the table via
GET_NHEL instead of copying the PROCESS_NHEL common raw -- otherwise an early
caller (reweighting builds its per-config helicity map at init, before any
matrix-element evaluation) would read the not-yet-materialized table as zeros.
GET_NHEL is defined by every standalone matrix.f, so this also links for
split-order processes (which have no FILL_NHEL).
Validated: e+ e- > mu+ mu- and e+ e- > w+{0} w- |M|^2 bit-identical to the old
table; 39/39 standalone cross-symmetry tests (fortran + cpp + mg7; density +
crossing + polarization + f2py); reweight per-event weights bit-identical
(p p > t t~ mass reweight).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…crossing guard test_config: add 'zerowidth_external': True to the expected default-config dict. It is a deliberately-registered default option (madgraph_interface.py:3290), sibling to zerowidth_tchannel, so the loaded config now carries it. test_polarization_top_decay: ungrouped `output madevent` (supports_crossing=False) on the crossing-tagged polarized top decays hit the crossing guard. The test does not exercise crossing, so scope it off with --use_crossing=False on all 14 generate/add-process lines (same fix as the decay-chain/grouping group). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…oop-induced handling
The [...] syntax (NLO, loop-induced, loop) must produce the same output as a
no-crossing build. Enforce it at the generation chokepoint: in
MultiProcess.generate_matrix_elements a process carrying perturbation_couplings
now skips the crossing branch entirely, so not even its tree-level Born/real
sub-amplitudes are crossed. breaks_crossing_symmetry treats a perturbative
process as crossing-breaking too, and the cross-group detector checks it BEFORE
partition_crossing_classes (which would otherwise IndexError on a loop-induced
matrix element). Fixes test_loop_induced_ggh (output madevent for
g g > h [QCD] no longer crashes in the crossing PDG tables).
DecayAmplitude / DecayChainAmplitude are Amplitude subclasses that override
default_setup with their own key set and carry no crossed_processes; the
compute_widths and MadSpin decay paths reach the Stage-C crossing reconstruction
with such amplitudes. Guard every crossed_processes access on the dict key
('crossed_processes' in amp) rather than the amplitude type -- isinstance does
not protect since DecayAmplitude IS an Amplitude. do_compute_widths also exports
its internal 1->N decays with use_crossing off (they carry no crossing but the
ungrouped madevent exporter refuses a crossing-tagged process). Fixes
test_ML_check_cms_aem_emvevex.
reweight: with merge_crossing='record' the default now FOLDS crossed
subprocesses instead of generating them as separate dirs, which the reweight's
id_to_path flavor matching needs when flavor grouping is off / keep_ordering.
Invert the condition so --use_crossing=False is emitted exactly in the cases
main produced separate dirs, reproducing main's subprocess layout. Fixes
test_oneloop_reweighting.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…level ME p p > t t~ reweight keeps crossing enabled (merge_crossing='record' with flavor grouping on), so the production |M|^2 is evaluated against the crossing-aware/folded SMATRIX rather than the pre-crossing separate-dir ME. This reorders the helicity sum, shifting each per-event weight by O(1e-3) absolute (~1e-5 relative), just above misc.equal's ~1e-3 window. Keeping crossing is correct here since this is tree level; the values are deterministic run-to-run and the reweighted cross-section is still 235.28 pb. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
test_madspin_ON_and_onshell_atNLO (p p > t t~ [QCD], spinmode onshell) raised KeyError ((-81,81),(-6,6,21)) because MadSpin's NLO production ME is generated as the LO real emission p p > t t~ pert_QCD, so crossing (merge_crossing=record) folds q q~ > t t~ g into its partner q g > t t~ q and drops the standalone subprocess. Keep crossing active and teach the density path to reach the folded channel through the representative's crossing-aware SMATRIX. - decay.py: register each matrix_element crossed_processes entry into all_me against the representative pdir (base wins; re-attach base decay chains for the full ME), so the onshell/density lookup no longer misses. - export_v4.py (write_f2py_splitter): append each subprocess' self-contained f2py_matrix_wrapper.f (PY_<prefix>GET_PDG_FOR_FLAVOR / GET_FLAVOR_LAYOUT / GET_NHEL_IDX / GET_DENSITY_IDX) into the combined f2py_wrapper.f. The combined all_matrix module was base-only; these per-process extended-FLAV_IDX entry points are the only way a python caller reaches a folded crossed subprocess. Concatenated into the single scanned source (adding them to the f2py -c line leaves the symbols undefined at dlopen on macOS). - interface_madspin.py: _build_cross_resolve() enumerates every (crossing,flavor) via the per-prefix layout/pdg accessors, keyed by sorted signed-physical PDGs; get_pdir resolves a crossed event to (representative prefix, extended FLAV_IDX); get_iden uses GET_NHEL_IDX for the crossed averaging denominator; get_density routes to GET_DENSITY_IDX with momenta in the crossed leg order (no merge revert). Base (uncrossed) behavior is unchanged. - amcatnlo_interface.py: skip generation-time-only options (zerowidth_tchannel) when replaying the MG5 set-history into the run interface, so the interactive launch does not trip check_set's run-time guard (a separate pre-existing crash unmasked once MadSpin succeeds). Validated: direct u u~ > t t~ g |M|^2 equals the folded-crossing FLAV_IDX result to 10 sig figs with the correct iden; c c~ / d d~ identical (massless); test green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…pickles test_aloha: the python writer now emits the t-channel width conditional (4ec2ae7) -- an `if (P**2).real > 0` around the width-kept denom with an `else` that drops it (M**2). Update the inline expected routines of test_short_pythonwriter{,_C,_4_fermion} accordingly (goldens). test_short_{sm,mssm,heft,sqso}: refresh the reference pickles (mg5_short_paralleltest_*.pkl). The large shifts are the deliberate t-channel width drop -- w+ w- > w+ w-, u u~ > z u u~, u u~ > d d~ w+ w-, h h > w+ w-, d d~ > x1+ x1- g -- and the sub-1e-5 shifts are helicity-summation float reordering from the NHEL encoder; no unexpected jumps or sign flips. Values regenerated from the current (canonical) matrix elements. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…uges compare_gauge.py checks that the complex-mass and fixed-width schemes agree in the unitary/Feynman/FD gauges. The default t-channel width drop (aloha.t_channel_width / zerowidth_tchannel) keeps i*M*Gamma for timelike but drops it for spacelike propagators; that s/t imbalance is an O(Gamma) gauge-invariance violation the FD gauge is sensitive enough to fail on (test_gauge_4_e500: e+ e- > e+ ve d u~). Make the comparison treat the width the same way in every variant: - 90 GeV runs (on the Z pole): keep the width, t-channel included (set zerowidth_tchannel False), so resonant propagators stay regulated. - 500 GeV runs (off resonance): zero every width in the fixed-width runs, so the three fixed-width gauges agree exactly. The complex-mass runs always keep their widths (the width defines that scheme). Also fix set_onshell_particles_width_to_zero: when a gauge boson is an external on-shell state it zeroed the internal gauge-boson propagator width but left the associated Goldstone (e.g. G+ <-> W) at its M*Gamma, a W/Goldstone inconsistency that breaks the Ward identity in Feynman/FD gauge. The matching Goldstone (keyed on its shared mass) is now zeroed too. Verified: test_gauge_4_e500 4/4; test_gauge_3 9/9 at both 90 and 500 GeV; w+ a > w+ a now emits ZERO for the Goldstone propagator width. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…modes test_short_madspin_singletop (p p > t j, t > b w+, w+ > l+ vl) failed on the crossing branch because crossing (merge_crossing='record', default-on) folds the t-channel u b > d t onto its s-channel partner q q~ > t b~ and drops the standalone subprocess. MadSpin's PDG-dispatch ME evaluators cannot reach the folded channel, breaking three spinmodes with distinct symptoms: - full_decay_chain (madspin_v1): KeyError ((5,81),(6,81)) in load_event -- the standalone_msP fortran (GET_FLAVOR_MS_PROD / FLAV_TABLE) is base-only, so the folded t-channel has no directory and no flavor index. - onshell_decay_chain (onshell_v1): ZeroDivision -- calculate_matrix_element (smatrixhel dispatches on concrete PDGs) returns 0 for the folded production. - madspin_density (density): NaN weights -- the reshuffle denominator MEdenom_prod = calculate_matrix_element(production) returns 0 and, being 0 (not None), overrides the correct crossing-aware density diagonal. Fix (keep crossing for the density modes; disable it only for the legacy PDG-dispatch modes): - decay.py: add a _no_merge_crossing() context manager (MG_MERGE_CROSSING=off escape hatch). Wrap the production AND full-ME generates in decay_all_events.generate_all_matrix_element (both must be wrapped -- the full-ME decay-chain generate also folds, and wrapping only the production leaves most events undecayed). Wrap the onshell-class generate only when mode=='onshell', leaving mode=='density' crossing on. - interface_madspin.py (calculate_matrix_element): when the base smatrixhel returns 0 on a crossing-folded event, fall back to the crossing-aware PY_<prefix>SMATRIX_IDX with momenta in the physical crossed leg order. Base events keep their non-zero smatrixhel value, so the fallback never fires for them. This fixes MEdenom_prod in the density path. Validated: singletop + ttbar + zz factory tests (5 spinmodes each), the NLO onshell/density crossing test (p p > t t~ [QCD]), 23 MadSpin unit tests and the 7 acceptance MadSpin tests all pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The decay-width comparators generate a 1->N decay (e.g. `t > all all`,
`z > e+ e-`, and the compare3 `part > m1 m2 m3 $ all $$ ...` form) and export it
with `output madevent`. That ungrouped exporter (supports_crossing=False) refuses
a crossing-tagged process, so with crossing on by default do_output raised
InvalidCmd from _check_crossing_support before any width was computed -- failing
test_decay_{sm,heft,mssm,nmssm1,nmssm2,nmssm3}. These are pure decays that carry
no crossing, so emit --use_crossing=False on the three generate sites.
(The "Command calculate_width not recognized" line in the logs is an unrelated,
pre-existing no-op print -- that command was renamed to compute_widths long ago
and is not the failure.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…FSy) combine_name's cross-base scheme (FFV2_FFS3) unconditionally appended the %(propa)s placeholder. For amplitudes (outgoing=0) the FLV_Coupling 'M' flag lives in %(tags)s, not %(propa)s (which HelasAmplitude fills with ''), so matrix.f called FFV2_FFS3_0 while ALOHA emitted FFV2_FFS3M_0, producing an undefined-symbol link error for processes like vt vt~ > ta+ ta-. Mirror the same-base scheme's outgoing-conditional placeholder: %(propa)s for wavefunctions, %(tags)s for amplitudes. No-op for non-FLV models (tags=''). Fixes the 6 tau/vt processes in compare_gauge test_gauge_2. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ing=False placement
TestMLMReweight (test_mlm_rewgt_*): generate_process() does `generate <proc>` then
`output madevent`; the ungrouped exporter refused the crossing-tagged process
(_check_crossing_support). These MLM-reweight cases don't exercise crossing, so
emit --use_crossing=False in generate_process. Fixes test_mlm_rewgt_{DY_uu_to_Z_jjjj,
gg_ttgg,qq_to_qq_schannel,qq_to_qq_tchannel}.
reweight_interface: the earlier reweight crossing fix inserted --use_crossing=False
right after the `add process`/`generate` keyword. For an NLO / EW-Sudakov LO
definition (`... [LOonly=QCD] --ewsudakov`) the flag then landed before the process
and was parsed as a particle ("No particle --use_crossing=false in model"),
crashing test_ttbar_ewsudakov. Append the flag at the END of each TREE process
definition only; perturbative ([...]) definitions are left untouched -- they
already skip crossing at generation. test_mass_reweighting / test_oneloop_reweighting
still pass (flag now correctly trailing, e.g. `generate p p > h j --use_crossing=False`).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…a flag The crossing guard refused an output that cannot read folded crossings whenever crossing was *requested*, regardless of whether anything had been folded, and it ran in the exporter constructor -- before generate_matrix_elements, so no recovery could run. Meanwhile the recovery already existed and its own comment called it "the safe default for any format that does not implement folding" -- but it was wired only into the GROUPED branch. Ungrouped output could therefore only ever error out. That is why ~40 tests carried --use_crossing=False for processes that fold NO crossing at all (w+ > all all, e+ e- > e+ e-, u u > u u, q q~ > q q~, u q > z u q, the sextet and polarized-decay processes: all zero). - Hoist the expansion into two shared methods: _crossing_needs_expansion() (data-driven: non-folding format AND crossings actually recorded) and _expand_recorded_crossings(). - Wire it into the ungrouped path via _expand_crossings_for_ungrouped_output(), including decay chains -- those record their crossings on the INNER amplitudes, so a naive key check would silently drop 4 crossings from p p > w+ p, w+ > l+ vl. - Restore the diagram-count ordering after regenerating decay chains: the regeneration walks _curr_proc_defs and lost the sort that decides the subprocess group numbering (P1_/P2_), renaming directories. - _check_crossing_support no longer raises (debug log): expansion is now the universal fallback, and the raise was flag-based and plainly wrong for a process that folds nothing. Verified byte-equal layouts, ungrouped madevent, crossing on vs --use_crossing=False: p p > j j QCD=0 (3 folded) -> same 4 dirs; p p > w+ p, w+ > l+ vl (4 folded, decay chain) -> same 6 dirs. Consequently reverted 41 --use_crossing=False from the tests and, more importantly, from the product code: MadSpin's decay-ME generation and do_compute_widths no longer touch crossing (both are pure 1 -> N decays that fold nothing). Kept in only three places, with honest comments: test_standalone_flavor_mask and test_standalone_wwjj pin the UNFOLDED layout because they open one specific subprocess directory that standalone legitimately folds, and the consistency suite uses the uncrossed build as its reference. Added test_standalone_crossing_folds_qqx_subprocess as their crossing-on counterpart (folded away, strictly fewer dirs, still reachable through a base that emits APPLY_CROSSING and demoes the quark initial state), and replaced the obsolete test_unsupported_output_raises_with_crossing -- it asserted an error message for u g > u g, a process folding nothing -- with a check that such an output is accepted and yields the same subprocesses. reweight_interface keeps disabling crossing for now (its id_to_path cannot reach a folded crossing, so a crossed flavor's weight was silently dropped); teaching it the crossing-aware f2py entry points, as was done for the MadSpin density path, is the remaining piece. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…t its verdict The C-parity helicity de-duplication could run on a pairing that had never been verified, undercounting the cross section: test_madevent_flavor_zud_nogroup gave an erratic 1.28 / 0.77 pb instead of ~3.215. read_good_hel forces NTRY(:) = MAXTRIES + 1, while the gate was DEDUP = NTRY(IFLAV).GE.20 and CSYM was a SAVEd local defaulting to .TRUE. So in any job restoring the good-hel file (refine, later runs) the IF (NTRY.EQ.1) reset never fires, the validating scan IF (NTRY.LT.20) never runs, and the reuse switches on immediately with every pair assumed C-symmetric -- including the parity-violating ones whose |M|^2 genuinely differ. Hence the erratic result rather than a clean factor. When the scan does run it is exact: for u d > z u d chirality leaves 2*2*3=12 non-zero rows and the scan invalidates precisely those, the 36 survivors being identically-zero rows where the reuse is a no-op. Changes: - CSYM becomes a single per-flavor verdict instead of a per-helicity flag. It is cleared as soon as ANY row fails to pair up (a self-paired row, FLIP(I)=I, has no distinct partner) or ANY pair mismatches. The reuse therefore always halves the whole loop, so the per-diagram AMP2 accumulated inside MATRIX and XTOT = sum_J AMP2(J) scale by the same 1/2 and the multi-channel weight AMP2(config)/XTOT is preserved by construction. A partial de-duplication would have rescaled AMP2 non-uniformly across diagrams. - Add NCSCAN, incremented only on passes that actually ran the scan: NTRY is unusable as a gate since the restore inflates it and the random-helicity branch bumps it without a full sum. - Persist the verdict next to the good helicities. write_good_hel emits it (conservatively "broken" when the run never completed its scan) and read_good_hel inherits it, defaulting to "broken" on a missing or old-format entry, since the good-helicity gate makes the scan irreproducible once restored. - CSYMBAD/NCSCAN live in COMMON/BLOCK_CSYM (zero-initialised, the convention NTRY already relies on) so the shared helper can reach them; sized flat MAXFLAVPERPROC*MAXSPROC to serve grouped and ungrouped alike. Applied to madevent (ungrouped and grouped), standalone, standalone_cpp and standalone_mg7. In C++ the good-helicity consistency check is folded into the flavor verdict: a pair split across the good/dropped boundary would otherwise lose the skipped row. The grouped 'CSYM PAIR:' report consumed by gen_ximprove/hel_recycle stays one-shot via NTRY, as NCSCAN saturates at 20. Note this switches the optimization off for parity-violating processes and keeps it for parity-conserving ones; the saving it gives up on chiral processes is redundant with the good-helicity filter, which already drops the zero rows entirely rather than half of them. Validated: test_madevent_flavor_zud_nogroup 3.171 pb (ref 3.215) with correct event flavor ratios, grouped test_madevent_flavor_zud still passing, test_standalone_merged_flavor_uq_zuq and the madevent/standalone consistency tests green, test_standalone_cross_symmetry 46/46, and standalone_cpp |M|^2 byte-identical before and after. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The reweight matched each event's flavor to a subprocess matrix element via
id_to_path, which is built from get_pdg_order -- and a crossing folded with
merge_crossing='record' has no get_pdg_order entry of its own: it lives at an
extended FLAV_IDX = cross*NFLAV+flav of its base. So every crossed event
silently lost its weight, and create_standalone_tree_directory worked around
that by force-appending --use_crossing=False to every TREE process definition.
Teach it to reach them instead; p p > w+ j, w+ > e+ ve goes from 6 subprocess
directories to 2, i.e. the crossed subprocesses now cost neither a generation
nor a compilation.
- madgraph_interface: new _crossing_folding_formats constant, replacing three
copies of the ('standalone','standalone_mg7') literal that had to agree, and
gaining 'standalone_rw' so the reweight's own output stays folded.
- f2py_splitter.py / f2py_wrapper_all.inc: new combined entry point
SMATRIXHEL_IDX(PROCINDEX, FLAV_IDX, ...), the same dispatch keyed on the
get_pdg_order slot with the extended index passed through -- the PDG dispatch
cannot name a folded subprocess and the FLAVOR array cannot express a
crossing. It shares f77_smatrixhel's alphas/scale2 block so that a crossed and
an uncrossed evaluation of one event use identical running couplings.
- export_v4: write_crossing_records emits SubProcesses/crossed_flavors.dat, the
CROSS codes each matrix element actually folded. This is required, not
bookkeeping: the runtime index space is dense and also holds crossings that
are merely applicable (a Z pulled into the initial state for p p > z j), so
matching an event against the raw enumeration would produce a wrong weight
rather than no weight. _crossed_signatures is refactored onto the shared
_recorded_crossing_matches, whose leg_matches is now symmetric (label-vs-label
as well as label-vs-member: it previously returned complete=False for every
decay chain, so check_sa fell back to the full loop) and which drops cross==0,
a recorded beam-swap mirror rather than a crossing.
- reweight_interface: build_cross_resolve walks GET_FLAVOR_LAYOUT /
GET_PDG_FOR_FLAVOR over the recorded codes only, applying each crossing to
EVERY get_pdg_order entry of the prefix -- a matrix element covers several
subprocesses in two independent ways, as flavor indices inside one entry and
as several entries the exporter combined (g u > h u with g s > h s), and
missing the second breaks every loop reweight, where grouping is forced off.
resolve_folded_crossing then picks the candidate per event from the signed
physical PDGs: within one merged tag g d > z d and g u > z u differ by ~25%,
so a representative flavor is not good enough.
The helicity dictionary of a crossed entry is the base one unchanged: SMATRIX
permutes its whole NHEL table before the helicity loop, so USERHEL=r selects the
configuration whose per-slot labels are base row r read positionally in the
crossed leg order. Verified per helicity against independently generated crossed
subprocesses on p p > z j, which unlike the W decay chain has enough non-zero
rows with distinct values to pin the mapping.
--use_crossing=False is kept for two modes, both commented at the site:
keep_ordering (its promise that events are written in the matrix element's own
leg order makes the lookup key order-sensitive, and a folded crossing has no
directory order to promise) and the density mode (it evaluates GET_DENSITY
through the FLAVOR array; GET_DENSITY_IDX would be needed, as MadSpin's density
path does it).
Validated: test_scan_reweighting (2362 of 5000 events resolve through the folded
path, correct flavor each time), test_mass_reweighting (235.28261285 pb
unchanged), test_oneloop_reweighting, TestMLMReweight 4/4,
test_standalone_cross_symmetry 47/47, test_standalone_madevent_consistency 9/9,
MadSpin spin_only + mixed_flavor_decay_log_summary. Independently, 400 events of
the test sample evaluated through the folded build and through a
--use_crossing=False build agree to 1.8e-16, 393 of them bit-identically.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
GET_CROSS_PERM accepted two families of CROSS code that are not crossings of the process, and the matrix element then returned a plausible-looking non-zero value for them rather than nothing. Any consumer that enumerates the index space -- MadSpin's _build_cross_resolve, the reweight's, check_sa -- could pick one up and believe it. - A leg conjugated WITHOUT changing side. SWAP_LEGS always negates both SGN entries, so XI==2 / XJ==1 (swapping the two initial legs of a 2->N: the beam swap, which must not conjugate) conjugated both of them. The existing overlap guard only fired when BOTH transpositions were active, so a lone one slipped through: p p > w+ j, w+ > e+ ve reported u~ g > e+ ve d, which does not conserve charge, at four times the base value. The same rule catches every XJ swap of a 1->N, where both partners are final. - A decay-block leaf carried across the initial/final line, which either splits a resonance or makes it initial: the same process reported g e+ > u~ ve d and g ve~ > e+ u~ d. compute_crossing_tables had always rejected these, and the comment in GET_IDENT_CROSS asserted "a crossing never moves a block leaf (GET_SPINCOL_CROSS rejects any that would)", but the generated GET_SPINCOL_CROSS rebuilt the permutation itself and had no such check. That duplicated rebuild is how the two drifted apart. Both rules live in the decoder now, which is the one place every consumer goes through, and are expressed with what is already there rather than a new table: the same-side test needs only NINCOMING, and the block test reads COUNTABLE, an NEXTERNAL-long per-leg list already emitted and DATA-initialised inside GET_SPINCOL_CROSS but until now unused there. A rejected code returns FLAV_IDX=0, which GET_PDG_FOR_FLAVOR and GET_AMP already treat as "no flavor", so no caller needed a new convention. GET_SPINCOL_CROSS now delegates its decode to GET_CROSS_PERM instead of rebuilding PERM, which is what keeps the two in step from here on. Mirrored in compute_crossing_tables (python) and in cross_perm_ic (C++; same-side only, that backend emits no COUNTABLE and so has no decay-chain crossing). The per-CROSS spincol list stays python-internal and is still not emitted to either backend. This does not make "applicable" mean "requested": z d > g d is a perfectly physical crossing of p p > z j that is simply not a subprocess of it, and only generation knows the difference. A consumer must still intersect with the recorded crossings, as the reweight does. test_standalone_cross_symmetry 47/47, plus the reweight suites, unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…nfig A within-group crossing router (matrix<i>_router.f) forwards its call into the base SMATRIX. With helicity recycling that base is matrix<b>_optim.f, whose helicity configs are baked into the recycled HELAS calls: it takes no runtime NHEL, so it can apply the crossing's momentum permutation and the IC/NSF sign flips, but never the helicity permutation that matrix<b>_orig.f applies (CR<b>_APPLY_CROSSING_TABLE: NHEL(XK,XR) = NHEL_IN(PERM(XK),XR)). gen_ximprove then prunes that baked table to the good-helicity set. The base's set is not closed under the crossing's helicity permutation, so the routed process silently loses part of its helicity sum and the cross section comes out low -- with no other symptom: |M|^2 checked through matrix<b>_orig.f is exact, and the channel counts, leshouche rows, CONFSUB rows and PDF<->flavor pairing are all correct. gen_ximprove already guards exactly this for a Track B cross-group base: when crossgroup_helunion.dat lists permutations for a matrix index it bakes that base over every config and skips the C-parity de-duplication (whose |M|^2 identity is only established for cross 0). Track A was never wired into it. Record each router's base->base helicity permutation the same way, so the base's own P directory gets the file. An unresolved permutation (a crossing that is not helicity-bijective) now also widens to every config: matrix<b>_orig.f has a run-time escape for it (GHIDX=0 makes it compute every helicity), the baked optim has none. q q~ > q q~ (q = u d s c, apply_flavor_grouping=False, grouped madevent, iseed=33, nevents=2000): 3,739,510 -> 5,207,310, bit-identical to the same process generated with --use_crossing=False, per channel as well as in total (G1 0.13101E+06, G2 0.36085E+07 -> 0.50763E+07). Wavefunction recycling is retained: matrix2_optim goes from 28 to 30 HELAS calls to cover 16 configs instead of 6. Tests: test_flavor_grouping_consistency_mlm, test_flavor_grouping_consistency, test_decay_chain_symmetry_factor; test_standalone_cross_symmetry (47); test_standalone_madevent_consistency (9). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A within-group crossing router (matrix<i>_router.f) forwards its call into the
base SMATRIX, whose multi-channel block hardcoded its OWN CONFSUB row:
DO I=1,LMAXCONFIGS
J = CONFSUB(<base proc_id>, I)
IF (J.NE.0) AMP2(J) = AMP2(J) * GET_CHANNEL_CUT(P, I)
XTOT = XTOT + AMP2(J)
ENDDO
ANS = ANS*AMP2(CHANNEL)/XTOT
Two things are wrong for a routed call, and neither can be fixed alone.
GET_CHANNEL_CUT(P, I) is evaluated on the DEPENDENT's momenta, so I has to be a
config of the dependent's row -- walking the base's row pairs every amplitude
with a different config's cut. But AMP2 is filled by the BASE's diagrams at the
CROSSED momenta, so the slot holding the dependent's diagram m is the base
diagram carrying m's topology *under the crossing*, not the one the base's row
names (that is the diagram sharing the topology with legs left in place).
Switching the row alone would index the base's AMP2 with dependent slot numbers;
translating the slot alone leaves the cut mispaired.
So do both, composed. The base bakes, per crossing code, the CONFSUB row to walk
(XGROWT, its own proc_id by default) and the dependent-diagram -> own-AMP2-slot
map from _crossgroup_configmap (XGCFG, column picked by XGCOLT); IXROW/IXR are
resolved from CROSSUSE in the decode and the loop becomes
J = XGCFG(CONFSUB(IXROW, I), IXR)
through a new me_confsub_j hole in both the plain and the helicity-recycling
template. Slot 0 of every XGCFG column is 0, so a config this subprocess has no
diagram for still reads back 0 and is skipped as before. The router applies the
same map to CHANNEL before the call (XGCONF_<i>), since only it knows which
config the sampled channel came from.
Resolved from the crossing code through baked tables, not a common block set by
the router: madevent runs vectorised (IVEC/warps) and mutable shared state would
race. Keying on the crossing alone is safe because compute_crossgroup_routing
skips any group that has within-group routing, so a Track-A base is never also a
cross-group (Track B) base and no foreign crossing can reach the tables. Only a
clean permutation of the base's own diagrams is accepted, so a
_crossgroup_configmap fallback keeps the historical row.
Invisible at the default settings: get_channel_cut returns 1d0 immediately when
sde_strategy=1 .and. tmin_for_channel=-1, every cut is 1, and XTOT is just the
sum over the row's slot SET -- which both rows share. It only bites at
sde_strategy=2. It was never a cross-section bug either: the error was a
bijective relabel, so the weights still summed to 1 and only the per-channel
split (importance sampling) was wrong.
Validated on p p > j j, P1_gq_gq (dependent row [1,3,2] vs base row [1,2,3]) with
a driver calling SMATRIX2 under MULTI_CHANNEL at a fixed phase-space point, one
call per config, sde_strat=2:
--use_crossing=False 48.5728746 / 5.39698606 / 3.03580466
before 48.5728746 / 3.03580466 / 5.39698606 (2 <-> 3 swapped)
after bit-identical to the reference
At sde_strat=1 all three agree to the last ulp. A 20k-event grouped run at
sde_strategy=2 confirms it through the recycled matrix1_optim.f: per-config -25%
and +96% before, within 0.4 sigma after, total preserved throughout. Output with
crossing off is byte-identical.
Track B (cross-group) has the same mispairing and is NOT addressed here: its
dependent lives in another group, so its configs are not rows of the base's
CONFSUB and the base's GET_CHANNEL_CUT uses the base group's iforest in the
base's leg order while P arrives in the dependent's.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…l row too
Same defect as the within-group case fixed in the previous commit, in the other
routing path: a cross-group (Track B) dependent calls the base group's SMATRIX,
whose multi-channel loop walks the BASE's CONFSUB row while GET_CHANNEL_CUT(P, I)
is evaluated on the DEPENDENT's momenta -- so every amplitude is paired with a
different config's cut, and the AMP2 slot is the one the base's row names rather
than the one the crossed evaluation filled.
It cannot be fixed the way Track A was. The base's matrix<b>_orig.o and
matrix<b>_optim.o are SYMLINKED into every dependent P directory
(write_crossgroup_mk), so one binary serves them all and nothing per-directory
can be baked into it. Two facts make a linker-resolved fix work:
* GET_CHANNEL_CUT lives in genps.f, compiled per P directory, so inside the
shared object it ALREADY means the dependent's config I. Only the row was
ever wrong.
* LMAXCONFIGS is one global maximum (Source/maxconfigs.inc, symlinked), so the
DO I=1,LMAXCONFIGS bound is the same in every directory -- a dependent's
configs always fit the loop the base was compiled with.
So the base calls CALL XGROW<b>(CROSSUSE, XGJROW) and uses J = XGJROW(I), reusing
the me_confsub_j hole added for Track A (the two branches are mutually exclusive:
compute_crossgroup_routing skips any group that has within-group routing, so a
Track-A base is never a Track-B base). Each directory links its own XGROW<b>,
written by write_xgrow_routines into auto_dsig.f -- the one file emitted exactly
once per P directory, so a base serving several dependents in one directory still
gets a single definition:
* where the base is generated: the identity, XGJ(I) = CONFSUB(<b>, I). Only
cross 0 reaches it, its group having no within-group router.
* in a dependent's directory: XGJ(I) = XGCFG(CONFSUB(XGROWP(IXR), I), IXR) with
IXR = XGCOL(CROSS) -- the routed subprocess's own row, each diagram mapped to
the base AMP2 slot the crossed evaluation filled (_crossgroup_configmap, the
same map its auto_dsig already uses for DSIG_XGCONFIG).
write_super_auto_dsig_file takes group_number to look up self._crossgroup;
MadWeight has its own override and is untouched.
Validated with a driver calling the routed SMATRIX under MULTI_CHANNEL at a fixed
phase-space point, one call per config, sde_strat=2, on p p > j j P1_qq_gg
(routed to P1_gg_qq with FLAV_IDX=24, DSIG_XGCONFIG=[1,3,2]):
--use_crossing=False 0.133719967 / 0.237724386 / 2.139519475
before 0.133719967 / 2.139519475 / 0.237724386
after bit-identical to the reference
The base's own directory called natively is bit-identical before and after, and
the within-group fix stays bit-exact. A 20k-event grouped run at sde_strategy=2
puts P1_qq_gg back on the reference (1.4575e5/1.4362e5 vs 1.4511e5/1.4416e5,
against 1.3116e5/1.5459e5 before). Output with crossing off is byte-identical.
Also generated and compiled for p p > j j j (two Track-B directories, 16-diagram
maps). As for Track A this is invisible at the default sde_strategy=1, where
get_channel_cut returns 1d0, and it never biased the cross section -- the error
was a bijective relabel, so only the per-channel weight was wrong.
If one directory both generated SMATRIX<b> and routed to another directory's
SMATRIX<b> the two XGROW<b> bodies would collide; that collision already exists
for SMATRIX<b> itself, so this only warns and keeps the historical row.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
merged_particles is keyed by the POSITIVE merged code ({81: [1,2,3,4], ...}),
but orig_order carries the SIGNED leg id. A subprocess whose grouped legs are
all anti-particles -- g q~ > w+ q~, get_pdg_order [21,-81,24,-81] -- therefore
failed `any(p in relevant_merged for p in pdg)`, the -81 labels were handed to
the fortran as-is, and the flavor mapping resolved them to "no flavour":
SMATRIXHEL and GET_DENSITY both return an exact zero. Fix is abs().
The two call sites were copy-paste, which is how they came to share the bug,
so they are now one helper, _pdg_for_me_call(). That also makes the density
side unit-testable without building any fortran.
Reachability is the opposite of what the symptom suggests. The |M|^2 path
(ReweightInterface) has flavor grouping ON by default and hard-crashes with
"Invalid matrix element" on any grouped `p p > w+ j` reweight -- that is the
live bug. The density call site is currently unreachable, because
DensityInterface._reweight_use_flavor_grouping returns False unconditionally
(deliberately: grouping would sum subprocesses and break the density-matrix
interpretation), so density-mode rw_me is built per-flavor and
merged_particles is empty. Verified: an end-to-end p p > w+ j density reweight
gives byte-identical results with and without this change. Fixed anyway, so
the site is correct if grouping is ever enabled there.
Also:
- DensityInterface.calculate_matrix_element now refuses an identically-zero
density matrix instead of averaging it in. With matrix_normalisation on (the
default) the trace is zero, so such an event was poisoning the average with
NaN rather than merely diluting it.
- get_crossing_tag used two sequential `if`s where the second must be `elif`.
Harmless in practice (merged codes 81+ are never individual-PDG keys) but
the same class of latent sign bug; tightened.
Audited the rest of this file, MadSpin/interface_madspin.py and
MadSpin/decay.py for the same blind spot: all other membership tests already
use abs() or discriminate on the sign deliberately.
Tests, both confirmed to fail before this change:
- tests/unit_tests/various/test_reweight_interface.py, 7 cases on the shared
helper. Only the all-negative cases break; mixed-sign passes either way,
since one positive leg already satisfies any().
- test_cmd_reweight.py::test_reweight_merged_antiparticle_labels, end-to-end
p p > w+ j (13 s) over a new fixture whose 4 events put 3 in the affected
subprocess, in both initial-state orderings, plus a g u > w+ d control.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…er_interface test_density_mode_user_interface was defined twice in the same class, so Python kept only the second definition and the first body had never run. The two bodies were duplicates: same process, same 50k events, same assertions and the same reference matrix. They differed only in where the mg5 command card was written (per-test tmpdir vs a hardcoded /tmp path), a stdout redirect, and a defensive parser branch for "np.complex128(...)" values that can no longer occur since the writer casts every entry to a plain complex. Keep the first (tmpdir-based) copy and remove the shadowing one, so the surviving test is the one that actually runs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Interaction.check_flavor mixed two conventions: line 1111 and 1116 tested
`abs(pdg) in model['merged_particles']`, but the guard between them used a
bare `pdg in model['merged_particles']`. 'merged_particles' is keyed by the
positive merged code only ({81: [1,2,3,4], 82: [11,13], ...}) while pdgs
holds the signed code, so the bare test is False for an interaction whose
merged legs are all antiparticles. Control then fell to the
`elif flavor == [0]*len(pdgs)` branch, which is also False (flavor was built
with abs() and is non-zero), and the code hit `raise Exception`.
Same sign mismatch as 82a8194ce, one layer up.
Replaced by a single `positions` list computed once with abs(); the guard is
now `if positions:`, so the two tests cannot drift apart again.
Reachability: negative merged legs are common (sm has [-83, 15, 24];
MSSM_SLHA2 has 90 squark-quark-gaugino vertices carrying -81), but the branch
sits inside `isinstance(coupling, str)`, and after merge_flavor the only
str-coupling interactions with merged legs are the flavour-diagonal ones
restored at base_objects.py:1660-1671 -- always particle+antiparticle pairs
([-81,81,21], [-82,82,22], [-82,82,23], [-83,83,23]), which always have one
positive leg. So sm and MSSM never hit it; the bug is latent.
It becomes a hard raise for a lepton-number-violating vertex, where both
merged legs are antiparticles: H-- l+ l+ -> pdgs [-9000005,-82,-82]. Such a
vertex passes the 2-fermion support_flavor filter, and its universal coupling
makes the restore-to-str step fire. Reproduced by driving the real
merge_flavor([11,13]) over sm plus a hand-added doubly-charged scalar: it
raised pre-fix and returns True/True/False for e+e+ / mu+mu+ / e+mu+ after.
`len(positions) != 2: raise` stays a sound invariant -- the restore-to-str
step requires `len([k for k in fkey if k]) == 2`, so restored str couplings
carry exactly 2 merged legs and untouched interactions carry 0.
Validated: 142 unit tests in test_base_objects / test_helas_objects /
test_process_checks pass; end-to-end grouped generate (p p > t t~, p p > z j,
p p > e+ e- j) and a standalone output are clean. test_model_equivalence's
test_sm_equivalence failure is pre-existing and byte-identical with and
without this change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…icore mode do_reweight has two out-of-process paths. The multicore one splits the LHE into <events>.lhe.gz_<i>.lhe chunks and starts one reweighting job per chunk, so in density mode each job wrote the average density matrix of its own chunk (Average_density_matrix_unweighted_events.lhe.gz_0.txt, ..._1.txt, ...). The mother interface never combined them, so the canonical Average_density_matrix_unweighted_events.txt that users and test_density_mode_user_interface expect was never produced, and the per chunk files were left behind as clutter after lhe.remove(). Recompute the average from the recombined event file instead: every event carries its own <density> tag, so the result is exact. - matrix_normalisation True (the default): the stored event.density is already trace-normalised, so rho_avg = sum(density * wgt) / sum(wgt). - matrix_normalisation False: event.density is the raw matrix and the writer accumulates a direct sum, so rho_avg = sum(density) / nevents. matrix_normalisation is read back from Cards/reweight_card.dat through the same parser do_change_matrix_normalisation now uses, so the two cannot drift. The "compute rho_avg, build the square matrix, write the .txt" block moves out of DensityInterface.launch_actual_reweighting into module level helpers, so the in-process writer and the multicore mother share one implementation and the file format stays byte for byte identical (entries are still cast to plain complex on purpose, so the consumer parser never sees np.complex128(...) wrappers). Validated on g g > t t~ with 3000 events: the multicore run (chunks of 2500 and 500 events) recombines to 0.3679096682722819 / 0.1320903317277266 / 0.06343533326759013 / -6.146879680399204e-07+2.8602362333319078e-05j, bit identical to the single core run over the same events. Note that this path is currently only reachable with force_run False, which no LO entry point uses (bin/madevent, bin/generate_events and launch from mg5_aMC all pass force_run=True and therefore reweight in process), hence the bug going unnoticed. The new acceptance test drives it explicitly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ver crosses generate_subprocess_directory passed xgrow_map= to write_matrix_element_v4 unconditionally, but that keyword only exists on the grouped madevent exporter. LoopInducedExporterME overrides the method without it, so `output madevent` for a loop-induced process died with TypeError: LoopInducedExporterME.write_matrix_element_v4() got an unexpected keyword argument 'xgrow_map' leaving the run_card unfilled -- which surfaced as test_loop_induced_ggh failing on '%(nevents)s can not be mapped to an integer'. A loop-induced matrix element never crosses in the first place (perturbative processes skip the crossing branch in generate_matrix_elements, and breaks_crossing_symmetry treats them as crossing-breaking), so the crossing plumbing has no business reaching it. Pass the kwarg only when a crossed subprocess is actually routed to this base. Fixes test_loop_induced_ggh (acceptancetest_104). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
get_crossing_tag builds the list of crossable signatures with [tuple(int(x) for x in sorted(list(t[0]) + list(t[1]))) for t in self.id_to_path] which assumes every key is an (initial, final) pair of PDG lists. The NLO path also stores the virtual matrix element under ((initial, final), 'V'), so t[1] is the string 'V' and the sort compares a str against a tuple: TypeError: '<' not supported between instances of 'str' and 'tuple' That killed the whole reweight run, and the test then reported the missing weight it was really after (KeyError: 'MYNLO_nlo'). Only a plain (initial, final) pair can carry a crossing, so skip anything that does not have that shape instead of trying to sort it. Fixes test_nlo_reweighting (acceptancetest_102). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The crossing path runs the host good-helicity loop over cNGoodMaxCross and lets every SIMD lane evaluate its OWN crossing's ighel-th good helicity (cGoodHelOfCross), but the reported per-event helicity was still read from the union list cGoodHel[ighel]. The crossing-aware good-hel scan ORs over every valid extended flavor id, so the union is wider than any single crossing's list and the two stop agreeing even for the identity crossing: the event is written out with a helicity row the lane never evaluated. For p p > t t~ that put every q q~ > t t~ event on one of NHEL rows 1-8 (initial quark helicity +1), whose |M|^2 is exactly zero. Reweighting reads the LHE helicity and evaluates that single row, so it raised "Invalid matrix element" on the first quark-initiated event and aborted the whole run, leaving rwgt_1 unwritten -- test_mass_reweighting_mg7, i.e. the acceptancetest_mg7_reweight CI job. Add a host-only selected_hel_code_lane() that reads the row from the same per-crossing list the lane used, and call it from the two selection sites. Only the reported helicity was wrong: |M|^2 and the cross section never were. The non-crossing exporter path is untouched and stays byte-identical. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`selected_hel_code` re-encodes the helicity config of a CROSSED event into its
canonical mixed-radix code, but built its per-leg digit table `xhel_states` from
`get_helicity_states(False)` while the cHel/tHel table it reads those values out
of is emitted with `allow_reverse=True` (get_helicity_matrix, the AV "#569"
comment) -- which is also the order the fortran ENCODE_HEL STATES table uses
(get_helicity_encoder_dict: "so the value order matches get_helicity_matrix()
... (code==row)"). get_helicity_states REVERSES the list for an ANTIparticle
leg, so every antiparticle leg's digit came out off by one state.
For u u~ > t t~ legs 1 and 4 give (+1,-1), not (-1,+1), and all 16 rows
mis-encode. Only crossed events were affected: cross 0 short-circuits to
base_ihel+1 before ever touching the table, so uncrossed output -- and
--use_crossing=False output -- is unchanged.
|M|^2 and cross sections were never wrong; the damage is downstream. mg7's LHE
writer indexes the BASE helicity table POSITIONALLY (export_mg7 ships
get_helicity_matrix() as `helicities`, lhe_output.cpp reads row helicity_index
slot by slot), so a shifted code names a different config and reweighting
evaluates a helicity whose matrix element is 0 --
reweight_interface.calculate_matrix_element then raises "Invalid matrix element"
and aborts the whole run.
Validated at runtime against the fortran backend, not just compiled: check_sa
built with BACKEND=cppsse4 and driven through the real allselhel path
(UMAMI_IN_RANDOM_HELICITY as a CDF sweep -> UMAMI_OUT_HELICITY_INDEX, MG_SAMEMOM
so every event shares momenta), compared against SMATRIXHEL evaluated one
canonical code at a time at the same momenta and the same extended flavor id.
For p p > w+ j and its RECORDED crossing 20, the fix reports codes {20,23} --
both non-zero, frequencies 108/20 matching the fortran |M|^2 weights
1.317/0.243 -- where the old table reported {14,17}, both |M|^2 = 0. Same
verdict for u u~ > g g crossed to u g > u g.
The generated "COMPILE-CHECKED ONLY, NOT VALIDATED AT RUNTIME" caution is
replaced by what the check established, plus one limitation it turned up: a
crossing that lands a leg in a slot with a DIFFERENT number of helicity states
(a massive vector moved into a fermion slot) has no representable base row and
falls back to digit 0, mirroring the fortran ENCODE_HEL D=1 fallback. That can
only happen for a crossing that is merely applicable and never recorded by the
generation, which consumers must already exclude.
test_standalone_cross_symmetry 47/47 OK, test_export_cpp 5/5 OK.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tests/acceptance_tests/test_standalone_cross_symmetry.py (47 tests), tests/unit_tests/interface/test_reweight_density.py (8) and tests/unit_tests/various/test_reweight_interface.py (7), plus the three new acceptance tests in test_cmd.py / test_cmd_reweight.py, were not selected by any test_manager.py invocation in .github/workflows: 65 tests that never ran. Add 8 jobs, split by toolchain and by cost (each madevent job runs one or two full generations, hence one apiece): unittest_34 15 unit tests, no toolchain, ~0.05s acceptancetest_crossing_static 7 codegen/python-only tests acceptancetest_crossing_fortran 22 fortran standalone tests acceptancetest_crossing_cpp 15 standalone_cpp/mg7 + check crossing acceptancetest_crossing_madevent_labels 2 LHE helicity/colour labels acceptancetest_crossing_madevent_xsec 1 decay-chain xsec regression acceptancetest_crossing_reweight 2 folded layout + merged -81 labels acceptancetest_density_multicore 1 multicore average density matrix Every job that builds a backend pulls in restore-pip-cache: it installs meson/ninja and sets f2py_compiler, which the f2py-backed tests need. That is not optional -- test_manager.py exits 1 when a test is SKIPPED, not only when it fails, while its runner prints a bare "OK" -- so a missing toolchain shows up as a red job whose log claims success. Noted in a comment above the jobs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The only cross-section-level guard on the crossing router was the decay-chain one (p p > w+ j, w+ > j j). Everything else compares matrix elements, which is blind to a wrong crossed averaging denominator, multi-channel row or good-helicity union: those leave the per-flavor MEs agreeing while moving the integral, which is how the routed groups lost ~29% before the helicity union was fed to the Track-A routers. p p > t t~ j j is the sharp inclusive case. Flavor grouping collapses it to five subprocess groups, two of which (gq_ttxgq, qq_ttxqq) are served by a cross-GROUP router -- matrix<i>_router.f plus crossgroup_helunion.dat instead of their own matrix element, i.e. evaluated by another group's matrix element under a crossing over the union of both helicity sets. Nothing in the suite integrated that path. Integrate it twice at the same seed (default vs --use_crossing=False) and require agreement within max(1%, 3 sigma). The premise is guarded both ways: the default build must emit a router, the reference build must not, so the test cannot degenerate into comparing two identical builds. Measured: 416.6 +- 2.4 pb routed vs 413.7 +- 2.6 pb independent (0.8 sigma), ~40s per integration, 75s for the test. Added to the acceptancetest_crossing_madevent_xsec job next to the decay-chain test. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The de-duplication that halves the helicity sum -- pair every row with its
fully flipped partner, evaluate one of the two and count it twice -- shipped
across five backends with no test of its own. The only thing pinning it was a
set of refreshed density-matrix goldens in test_cmd.py, which say nothing about
the branch that actually broke: the verdict defaulted to "de-duplicate" while
read_good_hel forces NTRY past MAXTRIES, so the validating scan never ran and a
flavor whose pairs nothing had verified silently summed half its helicities.
Bracket the rule with two all-massless 2->2 processes:
u u~ > g g parity conserving -- every pair matches, the reuse ENGAGES
d u~ > e- ve~ V-A, maximally parity violating -- the flipped partner of the
surviving row is identically zero, so the all-or-nothing rule
must REFUSE the reuse for the whole flavor
and check both levels for each: the per-row premise (|M(h)|^2 vs |M(-h)|^2 via
SMATRIXHEL, with the codes coming from the process' own ENCODE_HEL so the
pairing is pinned to the canonical encoding rather than a guessed row index),
and the consequence (30 successive unpolarized SMATRIX calls at one point must
all give the first call's value -- the scan phase covers the first 20, the fast
phase the rest).
Non-vacuity checked, not assumed: instrumenting SMATRIX to print DEDUP gives
"on from call 20, CSYM true" for the QCD process and "never, CSYM false" for
the charged-current one, so each test really does cover its branch. Both were
bit-identical across all 30 calls; the 1e-12 tolerance is headroom for the
reassociation the halve-and-double introduces on other processes.
_phase_space had no state beyond self.energy, so its body moves to a
module-level _massless_2to2() that both classes use; the method delegates.
Full file re-run after the move: 52 tests, OK.
Added to the acceptancetest_crossing_fortran job (same backend, +10s).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The comment described a --use_crossing=False that d19891c removed, and all three of its claims are now wrong: the ungrouped madevent exporter no longer refuses a crossing-tagged process (_check_crossing_support only logs a debug line and the recorded crossings are expanded back automatically), nothing is being disabled, and the cases do not uniformly avoid crossing -- the fg_false modes pass group_subprocesses=True and ProcessExporterFortranMEGroup does support crossing, so those runs go through the router. Nothing to salvage: why no flag is needed is documented on _check_crossing_support itself, and the expansion is pinned by test_ungrouped_madevent_expands_folded_crossings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A cross-group (Track B) routed subprocess integrated far off and very
unstably while its base was fine: g g > t t~ u u~ + u u~ > t t~ g g at
3000 events, the routed P2_qq_ttxgg spreading 13.6% over five seeds
against 5.5% for the native build, and all five runs failing to reach
their event target.
Not the matrix element -- routed and native SMATRIX agree to nine digits
at a fixed phase-space point. _crossgroup_configmap was silently
returning the identity, so DSIG_XGCONFIG was never emitted and the
dependent's channel index reached the base's AMP2 unmapped, pairing every
channel's importance weight with the wrong amplitude.
The cause is a signature collision. _diagram_leg_subsets characterised a
diagram by the set of its propagators' canonical external-leg subsets
alone, and two of the 36 diagrams differ solely in the particle carried:
the ggg chain and the four-gluon vertex's auxiliary field route {1,2}
identically. That collapsed 36 diagrams onto 35 signatures, the base
lookup lost an entry, the bijection check failed and the whole map
degraded -- mis-pairing every channel, not just the ambiguous two (35 of
36 entries move once it is computed properly).
Key the signature on (leg subset, |PDG|) instead. |PDG| and not PDG,
because crossing a leg reverses the flow through the propagators on its
path and conjugates them, so only the magnitude survives the relabelling
this signature exists to be invariant under. Renamed to
_diagram_topology_signature, no longer being leg subsets alone.
Make the fallback loud while here. It was completely invisible: a
degraded map is still a legal bijection that merely samples badly, every
matrix element agrees to the last digit, and the only symptom is a slow,
unstable integral behind an error estimate that means nothing. Each bail
now warns, naming both matrix elements, the crossing and the reason.
Both crossing paths call this -- the within-group router (Track A) and
the cross-group auto_dsig fill (Track B). p p > t t~ j j was falling back
on Track A too (g Q~ > t t~ g Q~ <- g Q > t t~ g Q, the same collision),
but there the identity happened to be the correct pairing, so it was
harmless by luck rather than by construction. p p > j j was always clean.
Verified end to end over five seeds per build: the routed subprocess now
reproduces the crossing-off build on every seed, bit-identical at full
precision (106.7621201160 on the seed checked), against +82% and 13.6%
spread before, and no run fails its target any more. The base directory
is untouched (its generated fortran is identical between the two crossing
builds). No fallback warning fires across 12 config-map calls over six
process configurations, so the sharper invariant does not over-reject.
TestCrossingConfigMap was written first and was red on Track B. It judges
the map against a propagator walk recomputed independently inside the
test, so it pins the contract -- a config must be routed to the base
diagram that is this one crossed -- rather than the implementation, and
covers Track B, Track A, a p p > j j control against over-rejection, and
the warning itself. Bound to the existing acceptancetest_crossing_static
job, which needs no toolchain; it belongs there rather than in an
integration job because a mis-paired map leaves the cross section itself
correct and so cannot be caught reliably downstream.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Implementation:
helmap.
- [x] C symmetry of helicity are now checked compatible with helicity recycling
- [x] Block beam polarisation + EVA under crossing (clear crash rather than silent wrong results).
- [x] handled only for the production side in case of decay chain syntax.
Validation:
Known limitations / follow-ups
merge option